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Abstract 

It is shown that the fourth standard model (SM) family quarks can be observed at the Fermilab 
Tevatron if their anomalous interactions with known quarks have sufficient strength. 
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It is known that flavor democracy |1] favors the existence of the fourth SM family fermions 



The masses of these fermions are expected to be nearly degenerate and lie between 
300 GeV and 700 GeV. According to flavor democracy the fourth family neutrino should 
be heavy. In the framework of democratic mass matrix approach, small masses for the flrst 
three neutrinos are compatible with large mixing angles assuming that the neutrinos are 
of the Dirac type P]. Obviously, the existence of the fourth SM family leads to a lot of 
cosmological and astrophysical consequences (see for example |^). 

The experimental lower bounds on the fourth SM family fermions are as follows 100.8 
GeV for charged lepton, 45 (39.5) GeV for Dirac (Majorana) neutrino and 199 (128) GeV for 
"down" quark decaying via neutral (charged) current. On the other hand, the partial- wave 
unitarity at high energies leads to upper limit 7/14 < 1 TeV for heavy 



The fourth family quarks will be copiously produced at the 



11 



.HC 



ermions 

A M and the fourth 



family leptons will be observed at the future lepton colliders 

In principle, the Tevatron may also contribute to the subject. First, the fourth fam- 
ily quarks can manifest themselves indirectly due to the enhancement in the Higgs boson 
production [3, Q- Second, they can be produced directly via possible anomalous gqq^ 
interactions. It should be noted that the arguments given in [15] for anomalous interactions 
of the top quark are more valid for and dj quarks since they are expected to be heavier 



than the top quark. In our previous papers |l6, 




17| we have shown that the superjet events 



observed by the CDF [l^ llSl 120] could be interpreted in relation to the latter mechanism if 
one assumes, in addition, the existence of a new light scalar particle, decaying dominantly 
to r+r^ and/or cc. 

In this work, we consider the anomalous production of U4 and quarks at the Tevatron 
via the subprocesses gu{c) — > U4 and gd{s, h) — > ^4, respectively; followed by either SM or 
anomalous decays into the SM particles. 

We use the following effective Lagrangian for the anomalous interactions of the fourth 
SM family quarks jlfi| : 

L = '^e^geqAa^^qiF^" + '^gzq^cT^.vqiZ^" + ^^?.g4fx^.T'^g*G'r + h.c. (1) 

where F^^ , Z^^ , and G^'^ are the fleld strength tensors of the photon, Z boson and gluons, 
respectively; are Gell-Mann matrices; Cq is the charge of the quark; ge,gz, and gs are the 
electroweak, and the strong coupling constants, respectively, gz = ge/ cos 9w sin 9w where 
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FIG. 1: The decay widths for the fourth family quarks with mass a) niq^ = 300 GeV and b) 
niq^ = 700 GeV. Horizontal lines correspond to the anomalous decay modes. 

9w is the Weinberg angle. ti'^^z,g define the strength of the anomalous couplings for the 
neutral currents with a photon, a Z boson and a gluon, respectively; A is the cutoff scale 
for the new physics. 

In order to calculate the cross-sections and^ the decay widths, we have implemented the 
new interaction vertices into the CompHEP 21] package. We have used the parton dis- 
tribution functions CTEQ5L [22] at = m^^. In addition to the dependence on the 
fourth family quark masses, the decay widths depend on Ky for anomalous decays and on 
CKM matrix elements Vq^q for SM decay modes. This is demonstrated in Fig. ^(b) for 
rriq^ = 300 GeV (700 GeV). Since and quarks are almost degenerate in mass, their 
anomalous s-channel production cross sections will be of the same order for equal anomalous 
couplings. If the SM decay modes of the fourth family quarks are dominant, an investiga- 
tion of M4 quark is advantageous because of the clear U4 hW^ signature comparing to 
(^4 — i> tW~ — i> hW^W~ . If the anomolaus decay modes are dominant quark has a clear 
signature d^ —>■ bV {V = 7, Z, g) with b-tagging. Corresponding Feynman diagrams are 
shown in Fig. |21 

First, let us consider the process pp U4X bW~^X. Table IJ lists the cross sections 
for signal (with couplings k/A = 0.5 TeV~^ and 0.25 TeV^^) and the main background 
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u,c^^ d,s,b^. , ^d,s,b 



(a) (b) 

FIG. 2: Anomalous production of 54 quarks followed by a) SM decay of M4 quark and b) anomalous 
decay of ^4 quark where V = g, Z,'y. 

TABLE I: The cross sections for the process pp U4X bW~^X with k/A = 0.5 and 0.25 TeV~^, 
and the corresponding background pp bW^X without and with pT cuts. 



rriu^ = 300 (700) GeV 



no cut with cut 



k/A, Te\-^ 0.5 0.25 0.5 0.25 

as+B, pb 22.1(0.62) 5.7(0.21) 20.4(0.34) 5.2(0.082) 
as, pb 0.184 6.8 x 10^^ 

pp —>■ bW'^X. We consider two cases: without any cuts and with > 50 GeV for b-jets for 
two limiting m^^ mass values, namely, 300 GeV and 700 GeV. Numerical calculations were 
performed for \Vu^f,\ = (K/A)-TeV. This yields the dominance of SM decay mode (see Fig. 
nj. In order to estimate the observability limits for anomalous couplings k/A, we use the 
definition of significance 

SS = ^-^±^^^^7^^Rl^t (2) 



where e = 0.5 is the detection efficiency including b-tagging and BR = 0.2 is the branching 
ratio of e^Ve + p^^^i- We also require the minimum number of signal events to be 10 

and SS > 5. Assuming Lint = 10 fb~^ for integrated luminosity. We obtain following low 
limits on the anomalous coupling: k/A = 0.03 TeV^^ without cuts and k/A = 0.01 TeV~^ 
with p!^ > 50 GeV for m^^ = 300 GeV. Corresponding numbers for = 700 GeV are 
k/A = 0.33 TeV"^ and k/A = 0.12 TeV-\ respectively. 

The next process we consider is pp d^X qVX where q = d,s,b and V = g, Z, 7. 
The condition \Vd^t\ < (K/A)-TeV ensures the anomalous decay mode of to be dominant. 
In Table [nj branching ratios and total decay widths of d^ quark with m^^ = 300 and 700 
GeV are given for the anomalous coupling k/A = 0.5 TeV~^ and k/A = 0.25 TeV~^. The 
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TABLE II: Branching ratios (BR) and decay widths T for ^4 quark with the anomalous couphng 
k/A = 0.5 and 0.25 TeV^^. The last row presents the cross section for anomafous production of 
quarks. 



771^4, GeV 




300 




700 


Av/ii., le V 




u.o 


u./o u.o 


u.zo 




gd{s,b) 


31 


31 31 


31 


BR(%) 


Zd{s,b) 


1.9 


1.9 2.1 


2.1 




ld{s,b) 


0.17 


0.17 0.16 


0.16 


r, GeV 




1.75 


0.44 22.4 


5.6 


cr{pp d^X), pb 




21.4 


5.19 0.18 


0.077 


cr{pp b^X), pb 




2.72 X 10^3 




2.25 X 10-6 



calculated signal cross sections for c?4 quark are presented in the last row of the Table |n] 

Keeping in mind the assumptions for the decays of ^4 and quarks, one can differentiate 
between M4 and ^4 quarks by identifying the charge of the lepton from W decay. However, ^4 
and di quarks have the same final state signatures. For this reason we will double the number 
of signal events in our estimations below. The decay modes of d^ and ^4 quarks consist of 
two-jet, Z+jet and 7+jet. Even though the dijet mode is dominant, the extraction of the 
signal doesn't seem to be promising due to the huge SM background. For the Z+jet mode, 
again Z ^ qq is not promising due to the large background; BR{Z l^l^) reduces the 
number of events a lot; Z —>■ uu results in mono-jet final states but one cannot reconstruct 
rrid^. Hence, the optimum final state is 7+jet. The background for this process is also large 
but, it can be reduced if one uses the advantage of b-tagging. For this reason we consider 
the signal process pp — > d^X tr/X with the main background pp byX. 

For illustration, in Fig. Elwe present the invariant mass distribution of the background and 
signal events for rrid^ = 300 GeV, and two values of n/A (0.5 and 0.25 TeV"^). Obviously, 
from Fig. IHl the signal is quite observable. As k/A decreases and/or rrid^ increases the 
situation gets worse. In order to observe at least 10 signal events with SS > 5 at Lint = 10 
fh~^, the anomalous coupling should satisfy k/A > 0.08 (0.9) TeV^^ for m^^ = 300 GeV 
{rrid, = 700 GeV). 

In conclusion, the fourth SM family quarks could be observed at the upgraded Tevatron 
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FIG. 3: Invariant mass distribution of b-tagged jet and photon for signal (m^^ = 300 GeV, k/A = 
0.5 and 0.25 TeV^^) and background. 

depending on the anomalous coupling and the mass values. For Lint = 10 fb^^, U4 quark with 
mass 300 GeV and SM decay mode can be observed if n/A > 0.01 TeV~^. For = 700 
GeV, the lower limit on n/A is 0.12 TeV~^. On the other hand, quark with mass 300 
(700) GeV and anomalous decay mode can be observed if k/A > 0.08 (0.9) TeV~^. 

This work is partially supported by Turkish State Planning Organization under the Grant 
No 2002K120250 and 2003K120190. 
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